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ABSTRACT 

This paper first presents an overview of the concepts of assessment and evaluation in 
Open and Distance Learning (ODL) environment. The large numbers of students and 
numerous courses make assessment and evaluation very difficult and administrative 
nightmare at Distance Learning (DL) institutions. These challenges informed exploring 
issues relating to assessment and evaluation like the development and use of question 
bank and deployment of technology in assessment with emphasis on strategies for 
sustainability. Assessment in ODL, unlike in the Conventional system where students 
have a range of opportunities to demonstrate their learning periodically, depends on 
formal assessment tasks only. Thus, for validity to be high in ODL assessment, 
assessment must provide the appropriate situation possible for measuring the specific 
abilities being measured. Reliability is maximized when the assessment items are 
many; the items are not too easy or too difficult for the candidates; and the marking 
consistent. Evaluation depends on Tutor Marked Assignments (TMAs) and End of 
Semester Examination (ESE) in ODL system. Large question/item bank (with reliable 
and valid questions) from which items are picked to constitute test are needed. 
Attempts are made to provide argument for the use of computer as a viable option for 
sustainable assessment based on its sustainable features especially for ODL 
institutions with a large number of students' population. It is hoped that these 
strategies if adopted have the potentials to eliminate some of the challenges that ODL 
institutions are currently facing in assessment. It will also strengthen quality assurance 
mechanism in ODL institutions. 

Keywords: assessment; evaluation; question bank; technology; strategies; 
sustainability. 

INTRODUCTION 

Distance Learning (DL) is the fastest growing form of domestic and international 
education today. In addition to the challenges of assessment in conventional 
institutions, distance learners are spread out geographically - physically separated 
from each other and from the instructor. Moreover, class sizes for DL courses are 
generally larger than those for traditional classes. For instance, the National Open 
University of Nigeria (NOUN) has about 10,026 registered students in 2004 (pioneer) 
admission, 17,028 registered students in 2005 admission and in 33 Study Centres, the 
third admission exercise escalated to 47000 students. Also NOUN has 105 academic 
programmes, 1383 courses, 61 CDs and 63 courses online at the time of this study. 
These large numbers of students and numerous courses make assessment and 
evaluation very difficult and administrative nightmare at DL institutions. 
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Yet, given large class sizes and their physical separation from students, many distance 
learner instructors feel they have few options for assessing students' performance. 

This paper hopes to offer some possible solutions based on issues and insights in both 
methodology and technology for sustainability in assessment and evaluation in ODL. 

AN OVERVIEW OF THE CONCEPT OF ASSESSMENT IN ODL ENVIRONMENT 

Assessment is a systematic basis for making inferences about the learning and 
development of students. It is the process of defining, selecting, designing, collecting, 
analyzing, interpreting and using information to increase students' learning and 
development (Erwin, in Brown & Knight, 1994). In education, assessment may be 
thought of as occurring whenever one person, in some kind of interaction, direct or 
indirect, with another is conscious of obtaining and interpreting information about the 
knowledge and understanding, or abilities and attitudes of that other person 
(Rowntree, 1977). There are two distinct interpretations of assessment. These are: 

> Interpretation of assessment in terms of the routine tasks that students 
undertake in order to receive feedback on their learning and a mark or grade 
signifying their achievement. 

> Assessment as applied to processes at the institutional level, for example, 
programmes evaluation. (Biswas & Pradhan, 2002) 

The focus of this paper is on the first interpretation. Assessment in ODL, unlike in the 
conventional (f2f) system where students have a range of opportunities to 
demonstrate their learning periodically, depends on formal assessment tasks. The 
major characteristics of assessment in both settings are: 

For the conventional system, the learners: 

> are known through lectures, tutorials and individual consultations; 

> have a range of opportunities to demonstrate their learning -their interest, 
motivation, questioning and interaction; 

> have opportunities to diagnose their errors before they go for formal 
assessment task; and 

> assessment activities have some flexibility. 

On the other hand, ODL learners: 

> are at a distance; 

> rarely enjoy varied opportunities to communicate their learning; 

> depend much on formal task as they have less opportunity in which to 
diagnose their errors or mistakes before formal assessment task; and 

> hence, assessment must be thoroughly planned, communicated and 
managed. 

ISSUES RELATING TO ASSESSMENT AND STRATEGIES FOR SUSTAINABILITY 

A number of issues are relevant in assessment and are to be well thought - out in 
assessment processes in both the conventional and the ODL systems. These issues 
encompass: 

Stake holders in Assessment 

> Students need assessment to establish precisely: 

■ How they are progressing with their studies; 

■ Whether they are achieving the required standard; and 

■ To gain certificate of a level of achievement. 
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> Tutors need to ascertain: 

■ Whether students are attaining the intended learning outcomes; 

■ Whether course materials and distance learning activities are 
effective; and 

■ That learners have achieved standards or met requirements. 

> Institutions require to: 

■ Provide evidence of achievement of institutional aims; 

■ Determine whether programmes are effective in their stated aims; 

■ Certify that learners can practise in specific vocational areas; and 

■ Make judgement about admission to courses/programmes. 

> Community require to state whether: 

■ Institutions and tutors are effective and deserve continued funding; 

■ Students are adequately prepared for their careers; and 

■ Education is being geared to meet the broad, long term needs of the 
society. 

RELIABLE AND VALID ASSESSMENT 

Reliability and Validity are terms used for educational assessment 
Validity is the extent to which an assessment measure does the job for which it is 
intended; evidence that an assessment measures what it is assumed to measure. The 
validity of an assessment can only be determined by reference to the assessment's 
stated purposes and its designs specifications. For validity to be high the assessment 
must analyze student's performance on each objective and the assessment should 
provide the appropriate situation possible for measuring the specific abilities being 
measured. 

Reliability refers to consistency or precision or dependability of the assessment 
measurement; how consistent results are from one to another. The assessment is 
reliable if the results would be replicated on a subsequent occasion. Reliability is 
maximized when the assessment items are many; the items are not too easy or too 
difficult for the candidates; and the marking is consistent 

Methods of Assessment 

There are various methods of assessment. The commonly used ones are: 

> Objective questions. This method of assessment derives its name from 
objectivity employed in marking the answers to the questions. The marking 
which is a simple mechanical process can be done by an individual or by a 
computer. Examples of objective type questions are True/false items; Fill in 
the blanks item; multiple choice items; Completion items and Matching 
items. 

> Short answer questions. This may take different forms of answers such as 
one word/phrase; one sentence; completing a table/diagram; preparing a 
list or writing a paragraph. 

> Long answer questions. This may also take various forms such as essay; 
reports or dissertations. 

> Presentation. This method is used to assess presentation skills. It requires 
appropriate assessment criteria for assessing the appearance of presenter; 
introduction of self; introduction of presentation; content of presentation; 
logic and order of presentation; eye contact; audibility; handling questions 
and use of visual aids. This method of assessment has high validity when the 
presentation is assessed by a group. 
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ASSESSMENT IN OPEN AND DISTANCE LEARNING (ODL) 


Educators generally work with learners to review and support learning; and to make 
judgment regarding their merits and their achievements. In ODL, Self Assessment 
Questions (SAQs); Tutor Marked Assessments (TMAs) and End of Semester 
Examinations (ESEs) are used. There is no immediate feedback and reinforcement. The 
SAQs embedded in the course material enable distance learners to evaluate their 
progress frequently. The TMAs serve as the continuous assessment and the marks 
obtained carry about 25% to 30% weight which count in the final result of the course. 
The ESE carries about 70% to 75% weight age of the final results. Distance learners 
are usually free to appear at any of these examinations either for specific courses or for 
all the registered courses provided that the minimum period of study prescribed for the 
relevant course is completed. Like in the conventional system, assessment in ODL can 
be both formative and summative. Formative assessment takes the form of tutor 
marked assignments (TMAs). This is exemplified in the assignment responses of the 
learner and the writing of tutors' comments on them. It helps to identify the 
weaknesses and strengths in learning. It also helps to improve upon the process and 
the attainment of learning. Grading or scoring may be done in formative assessment. 
Summative Assessment takes the form of 'ESEs'. Both TMAs and end of semester 
assessment grades lead to final award of a certificate/diploma/degree to the learner. 

THE CONCEPT OF EVALUATION 

Evaluation is the systematic process of collecting, analyzing and interpreting 
information to determine the extent to which pupils are achieving instructional 
objectives. (It answers the question "How good?"). Evaluation may pertain to either or 
both of quantitative description (measurement e.g. testing) or qualitative description 
(non-measurement e.g. observation) of pupils. It always includes value judgment 
concerning the desirability of the results or to judge the effectiveness of a teaching 
programme. The emphasis of evaluation is on the extent to which learning outcome is 
achieved. It subsumes the roles of measurement and non-measurement techniques. 
Hence, it is comprehensive in nature (Gronlund, 1985). Evaluation in education is a 
systematic activity involving: 

> The analysis or documentation of programme related components and 
processes; 

> The measurement of variables associated with the programme; 

> The elaboration of recommendations based upon the collected information; 

> The activities provide opportunities for feedback, accountability and cost - 
benefit information (Biswas & Pradhan, 2002). 

Evaluation in Distance Education 

Here in the National Open University of Nigeria, evaluation depends on Tutor Marked 
Assignments (TMAs) and End of Semester Examination. In ODL system, the individual 
learners proceed at their own paces and sit for examination when they are ready. This 
results in provision for individual assessment/evaluation. 

Thus, large question/item bank (with reliable and valid questions) from which items 
are picked to constitute test are needed. Assessment differs from evaluation in the 
following ways: 

> Assessment focuses on the learning of the students while evaluation focuses 
on the way the various components of a course perform; 

> Assessment focuses on the performance of the students (grading or 
marking) while evaluation focuses on the performance of the provider and 
the provision; and 

> Assessment result may be used as a source of information while evaluation 

results have no direct bearing on students (Biswas & Pradhan, 2002). 124 


Evaluation is a broad term. Hence, this paper focused on assessment of students 
learning outcomes. 

QUESTION BANK 

A question bank is a planned library of test items pooled through cooperative efforts 
under the protection and support of an institution for the use of evaluators, academics 
and students in partial fulfilment of the requirements of the teaching learning process 
(Biswas & Pradhan, 2002). 

It is designed to fulfil certain pre-determined purposes namely to enrich the 
instructional aspect and to judge the distance learner. It offers a utility service with an 
inbuilt feedback mechanism for improvement of its questions. 

The importance of question bank for sustainable ODL 

The number of questions needed for a course/prog ram me which runs for a number of 
years is very large in a distance education institution. The academic staff are 
accordingly supposed to prepare a large number of quality questions on different topics 
of the prescribed courses/programmes for learners self assessment, TMAs and ESEs. 
Most of these academics lack the adequate skill in test development and the time at 
their disposal is limited and they cannot be expected to develop a question pool of 
reasonable magnitude and quality within a specific time. 

In the absence of a stock of ready made questions, the quality of question papers is 
liable to suffer. This necessitates the entrusting the preparation of quality items in 
different subjects to experienced teachers who are well conversant with the content 
and techniques of framing questions. 

This strategy helps to build up ready made stock of items for learners self assessment 
(SAQs), TMAs and ESEs. Such pool of item is of immense use if developed according to 
predetermined objectives. 

The type of questions making up a question bank are to depend entirely on the total 
framework of reference envisaged at the planning stage - SAQs, TMAs, ESEs items. 

Planning a question bank 

Running a question bank efficiently to sustain assessment necessitates careful 
planning. The objectives of such a bank should clearly be visualized and include: 

> To increase the value of measurement; 

> To increase the pedagogical value of evaluation; and 

> More detailed specific objectives on special need for the question bank 

The question bank should be located in such a place that it should be managed properly 
to provide utility services to all those who are interested. The planning of a question 
bank should take care of: 

> suitable place and equipment for storing; 

> Vetting and screening of items before depositing them in the bank; 

> Sorting and classifying items in the question bank; 

> Arrangement for subject wise assistance for efficient maintenance and 
up keep; and 

> An academic manager for question bank to direct the activities of the 
question bank 
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Development and use of question bank 

A good question pool is one that contains questions on all the topics in a course testing 
all abilities in a pre-determined structure. The generation of sets of questions for the 
question bank is based on test blue prints and question paper design. The design of a 
question paper considers setting percentage weight to these parameters: 

> Learning objectives; 

> Types of questions; 

> Content areas in a course; and 

> Difficulty level.(Prasad & Xavier; 2006) 

Based on these parameters a blue print / table of specification is designed. A blue print 
is a two dimensional table that specifies the level of objectives (expected learning 
Outcomes) in relation to the content (modules/units) of the course. A blue print 
defines how many items are to be taken from each content -module/unit, for each level 
of learning objectives and for each type of item. A detailed table of specification helps 
to ensure representative sample of items for a course. 

A Sample of a hypothetical test blue print/table of specification 


Content 

Objectives 


Long 

Answer 

Items 

Area 

Weight 

Knowledge 

10% 

Comprehension 

15% 

Application 

15% 

Analysis 

30% 

Synthesis 

10% 

Evaluation 

20% 

Total 

Module 1 

15% 

— 

1 

- 

2 

- 

- 

3 

l 

Module 2 

15% 

- 

1 

- 

2 

- 

- 

3 

l 

Module 3 

30% 

1 

- 

1 

1 

1 

2 

6 

2 

Module 4 

30% 

1 

1 

1 

1 

1 

1 

6 

2 

Module 5 

10% 

“ 


1 


- 

1 

2 

1 

Total 

100% 

2 

3 

3 

6 

2 

4 

20 

7 


The basic steps in preparing enhanced valid, reliable and useful items surround 
determining the purpose of testing, developing the test specification/test blue print, 
selecting appropriate questions, preparing relevant questions, assembling the test and 
using the results to improve subsequent items. Others are stating clear instructions to 
respond to the questions and sentence structure of a question. The use of test items 
appropriate for the outcomes being measured such as easy of scoring, ease of 
interpretation and application and cost of developing items (Biswas & Pradhan, 2002). 

Screening of questions/In-house moderation 

The written items are passed onto other members of the group for their comments. The 
comments are passed onto the item writer who in consultation with two or three 
participants finalizes the item. 

Also, the individual items may be written on the chalkboard by the author for the items 
to be discussed by the participants. This method of previewing item consumes time but 
it is educationally more potent and pedagogically sound. The process improves quality 
of items and provides good training to the participants for framing good items. 

Screening of questions/Expert moderation 

The second level screening may be done by a group of three experts all conversant with 
the techniques of test construction. These three experts may consist of: 


> A subject specialist who is in a better position to pass judgment on the 
authenticity of the subject matter. 
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> A tutor who has the experience of teaching that particular class for which 
the item is written and is in a better position to judge the suitability of the 
item for a given grade. 

> An evaluation expert who can help in improving the format of the question 
in the light of the objectives to be tested 

Using and updating the question bank 

Carefully selected questions from the question bank can be used for instructional 
purposes. Various types of questions may be selected from the question bank for pre¬ 
testing, review and revision of a lesson. A question pool from a question bank can be 
utilized in the preparation of textual material and review exercises in course materials. 
Learners can also use questions from question bank for self assessment of their 
learning outcomes for specific units or modules of a course since outline answers are 
provided in such questions. A question bank is evaluated at regular intervals at least 
once in three years, the questions are screened and the obsolete ones are discarded. As 
old and obsolete materials are discarded, new materials are continually added in line 
with the revision of course materials. Enrichment of questions by updating, replacing, 
discarding, modifying, adding new ones, regrouping and classification is an ongoing 
process aimed at giving the question bank a dynamic look (Biswas & Pradhan, 2002). A 
well developed and efficiently used question bank is an asset to sustainable 
assessment. 

COMPUTER BASED ASSESSMENT (CBA) 

Computer based tests are defined as test or assessments that are administered by 
computer in either stand alone or networked configuration or by other technology 
devices linked to the internet or World Wide Web (WWW) (Biswas & Pradhan. 2002). 
Computer based testing is recognized as a sustainable strategy for sustainable 
assessment for ODL because: 

> It can be more responsive to the needs of both the test 
provider/institution and distance learners. An example is where "on 
demand" testing of examinees is needed; the use of computer based 
testing is more helpful; 

> Computer based testing works on a completely different model than 
paper and pencil administration and offers the benefits that the later 
cannot match; 

> Smaller number of candidates can be tested throughout the year rather 
than large numbers several times a year; 

> Study centres can offer different tests at the same time since 
examinations may be delivered on personal computers using local area 
network (LAN); and 

> The computer selects test questions from a pool, so candidates taking 
the same examination will not be answering identical questions, which 
would enhance test security. 

SUSTAINABLE FEATURES OF COMPUTER BASED ASSESSMENT FOR ODL 

Computer based assessment allows for a diverse range of question types which is a 
better test of candidates competency. It can be offered at different times, locations or 
even different tests to different students. The computer selects test question from a 
pool, so candidates taking the same examination will not be answering identical 
questions. It eliminates the need for test booklets and answer sheets that increase the 
security levels. Self paced tutorials show candidates how to use a mouse and other 
testing tools, ensuring that even those without computer experience are comfortable. 
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Candidates can use either paper and pencil or word processing for essay questions 
depending on their own preference. It improves the link between instruction and 
assessment, providing a profile of candidates' strengths and weaknesses, and matches 
questions and the order in which they are presented to the ability of each test taker. 

Computer based examination system reduces the large proportion of workload on 
examination, training, grading and reviewing. The set of questions mostly used in 
computer based examination system are multiple choice objective tests that can be 
formally and easily evaluated online. Emphasis is now shifting to computer based tests 
and on-screen marking of essay questions in the developed world. Computer based 
tests are scored immediately or shortly after administration unlike the paper and pencil 
test result that takes several months to process. Fast scoring helps for publishing 
results quickly. It can be used to control inadequate examination halls. Lack of admin is 
a dream and it is the way forward. 

Limitations of computer based testing attempts to grade students' essays online are 
now being explored in the on-screen marking is already being practiced by Cambridge 
Assessment. On-screen marking is much better than paper scripts. This process 
although already practiced by some institutions in the developed world is stilled at the 
fundamental stage. CBA necessitates that the question bank needs be large enough to 
prevent high levels of repetition. Computer crashes - regardless of the computer 
system in use occurs. Hence, there is need for contingency plan in case there are any 
technical faults in the middle of the examination period that may prevent student from 
completing the examination and disadvantage their results. Other limitations are 
related to resources required - a sufficient numbers of computers, a room to install 
them, appropriate software and adequate technology expertise. 

Implications of computer based assessment for ODL institutions 

The development and use of computer based assessment such as "On Demand 
Examination System" would reduce the problems associated wit the traditional 
methods. It is possible to space the period of examination without compromising 
quality and integrity of examination. The system has the potentials to reduce 
examination malpractice as examinees are duly authenticated before sitting for the 
examination. The examination can be scored immediately and comprehensively; and 
can be easily collated and released on time. More investment is needed in the areas of 
infrastructure and human development especially in the area of ICT training. It is easy 
to administer and the verification of result is ease and devoid of paper work. The long 
time involved in marking examination scripts which in most cases are prone to errors 
and misplacement of some scripts due to large volume of scripts that has to be marked 
and assessed can be managed. Only registered candidates of the institution having 
enrollment numbers and valid identity cards are eligible to appear in the examination. 
These processes enable ease processing of examination results. Computer based 
assessment is a viable sustainable assessment strategy that could reduce the burden of 
examination result processing already experience by ODL institutions in Nigeria. 

CONCLUSION 

Open and Distance Learning (ODL) is the fastest growing industry in education sector 
today. The process of assessment of students learning outcomes remains a big 
challenge to the validity and reliability of ODL provisions. It is imperative for ODL 
institutions to enhance the quality of their assessment practices by commitment to 
issues relating to assessment strategies for sustainability. The deployment of 
functional question banks and computer based assessment is needed to foster 
modernization of assessment programmes in line with practices in the developed 
world. 


128 


BIODATA and CONTACT ADDRESSES of AUTHOR 


Charity Akuadi OKONKWO (Ph.D) is an academic staff of 
National Open University of Nigeria. Presently, she is a 
Programme Leader II (Senior Lecturer) in the School of 
Education of the University. She has her specialties in Physics 
and in Education (Measurement and Evaluation). She holds 
M.Sc. in Physics and M.Ed, Ph.D in Measurement and Evaluation. 
She has been in teaching profession for more than twenty five 
years where she served humanity in various levels of education 
and community services and has contributed to growth of knowledge through her 
teaching, research and publications. Charity belongs to professional bodies like the 
National Council of Educational Psychologists and the Science Teachers Association of 
Nigeria. Her hobbies include sewing, gardening, reading, research and publications in 
teaching, learning and assessment of students learning outcomes. She hails from 
Anambra State of Nigeria and she is married with children. 

Charity Akuadi OKONKWO Ph.D 

School of Education 

National Open University of Nigeria 

14/16 Ahmadu Bello Way; P.M.B. 80067 Victoria Island, Lagos, NIGERIA 

GSM: +2348036668264 

Email: caeokonkwo@vahoo.co.uk 

REFERENCES 

Biswas, P. & Pradham, B. (2002). Assessment and Evaluation in Distance Education. 

STRIDE HANDBOOK 4. Staff Training and Research Institute of Distance Education - 
Indira Gandhi National Open University, Maiden Garhi, New Delhi-110068, India. 
http://www.iqnou.ac.in 

Calder, J. (1994). Programme Evaluation and Quality: A comprehensive guide to setting 
up an evaluation system, Kogan page ltd. London NI9JN. 

Erwin, in Brown, S and Knight, S. (1994). Assessing learners in higher education, Kogan 
page, London. 

Gronlund, N. (1985). Measurement and Evaluation in Teaching, fifth edition, Macmillan 
Publishing Co., INC, New York. 

Prasad, S.K. & Xavier, G. (2006). Use of ICT in Evaluation. On Demand Examination 
System of National Institute of Open Schooling (NIOS), India. Paper presented at the 
workshop on "On Demand Examination System" organized by the National Open 
University of Nigeria (NOUN), NOUN Headquarter Lagos, 6-17 November. 

Rowntree, D. (1977). Assessing Students: How shall we know them? Kogan Page Ltd., 
London 



129 




